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— The MAILING DATE of this communication appears on the cover sheet with the correspondence address — 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will appty and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

Responsive to communication(s) filed on 20 September 2005 . 
2a)S This action is FINAL. 2b)D This action is non-final. 

3) 0 Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) H Claim(s) 98-151 is/are pending in the application, 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) n Claim(s) is/are allowed. 

6) I3 Claim(s) 98-151 is/are rejected. 

7) Q Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 

Priority under 35 U.S.C. § 1 1 9 

12)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)D All b)D Some * c)H None of: 

1 Certified copies of the priority documents have been received. 

20 Certified copies of the priority documents have been received in Application No. . 

3.D Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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Art Unit: 2621 



DETAILED ACTION 



1. 



The response received on 09-20-2005 has been placed in the file and was considered by the examiner. An 



action on the merits follows. 



Response to Arguments 



2. 



The arguments filed on 09-20-2005 have been fully considered. A response to these arguments is provided 



below. 



Claim Objections 

Summary of Argument : 

Applicant traverses the objection to the claims on the ground that the claim makes grammatical sense as currently 
written. 

Examiner *s Response : 

The examiner agrees. The claim objection is hereby withdrawn. 

35 USC 112, First Paragraph Rejections (Priority to Continuation-in-Part) 

Summary of Argument : 

(a) Applicant alleges that the limitation "determining a step size" is supported by the specification. Specifically, 

applicant argues that this limitation is inherent in the following passage: 

These sample pixels may be generated by any of various sampling techniques, such as grid-based 
sampling, random sampling, or other non-uniform sampling" 

Applicant alleges that "determining a step size is inherent in grid-based sampling" and that the limitation is therefore 
supported. 

(b) Applicant alleges that the limitation "determining one or more dominant color categories" is supported by the 
specification. Applicant includes several passages from the parent specification in support of this argument (see 
remarks, pgs 17-18). 

(c) Applicant has failed to direct any arguments refitting the examiner's contention that the limitation "searching a 
proximal region proximal to the color match candidate area" is not supported by the parent specification. 

Examiner 's Response : 

(a) The examiner disagrees. The cited passage does not clearly support this limitation. The limitation in question is 
not inherent in the passage cited by the applicant. Indeed, it does not even appear to be particularly relevant. 

(b) The examiner agrees. 
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(c) Since no arguments have been advanced, the examiner will assume that this is an admission that the limitation is 
not supported by the parent specification. Accordingly, this limitation will not be given the benefit of that earlier 
filing date. 

Claim Objections 

3. The follow quotations of 37 CFR § 1 .75(a) provide the basis of objection: 
(a) The specification must conclude with a claim particularly pointing out 

and distinctly claiming the subject matter which the applicant regards as his invention or discovery. 

4. Claims 108, 128, 147 are objected to under 37 CFR § 1.75(a) as failing to particularly point out and 
distinctly claim the subject matter which the applicant regards as his invention or discovery. 

These claims all refer to "determining a sub-sampling size." The metes and bounds of this phrase as 
currently recited are unclear. The respective parent claims make no mention of a type of, or mechanism for 
sampling, or a sampling size. Thus, it is unclear what this sub-sampling size is, and how it fits into the context of 
the respective parent claims. 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set 

forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

6. Claims 98-103, 107, 108, 1 10, 1 1 1, and 1 14 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
the combination of Abdel-Mottaleb et al. (Abdel) (USPN 6,226,636) and Chen et al. (Chen) ("Similarity-Based 
retrieval of images using color histograms"). 

Regarding claim 98 : Abdel discloses performing a color characterization analysis of the template 
image to determine color features of the template image (Abdel col. 2 lines 10-19, inter alia: The reference 
describes performing a color characterization on a template image (i.e. one of the plurality of images used to build 
the database) to determine color features (Abdel col. 5 lines 55-64: This image is analyzed to determine color 
features). 

Abdel further discloses performing a color characterization analysis for a plurality of regions within 
the target image to generate color characterization information for each of the target image regions (Abdel 
col. 5 lines 27-33: The reference describes analyzing target images (i.e. query images) in the same color 
characterization manner.). 
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Abdel further discloses searching for regions of the target image having a color characterization that 
matches, at least to a degree, the color characterization of the template image (Abdel col. 5 lines 27-33: The. 
reference describes determining similarity between the template image (i.e. predetermined images) and the query 
image by matching regions of the query image.). 

Abdel therefore discloses a color characterization analysis performed for the template image and for a 
plurality of regions of the target image. Abdel, however, fails to disclose the details of this color characterization as 
required by the claim. Specifically, Abdel fails to discloses assigning pixels to an HSI color space. 

Chen, on the other hand, discloses assigning each of at least a subset of pixels to at least one color 
category that corresponds to a portion of a hue, saturation, and intensity (HSI) color space (chen pg. 644: 
HSV is equivalent to HSI). Chen further discloses determining information indicative of the allocation of the 
assigned pixels across color categories (Chen pg. 644-645: The reference describes color histogramming). It 
would have been obvious to one reasonably skilled in the art at the time of the invention to modify the target and 
template image comparison technique of Abdel by characterizing those images using HSI color information as 
taught by Chen. Such a modification would have allowed for an image characterization using an image space that 
was in accord with the sensitivities of human perception (Chen pg. 644, next to the heading "Q3"). 

Regarding claim 99, Abdel discloses generating information specifying a location for each region of the 
target image that matches the color features of the template image (Abdel col. 5 lines 46-54: The reference specifies 
a location for each of the N regions). 

Regarding claim 100, Abdel discloses that for at least one region of the target image that matches the color 
features of the template image, displaying information on a graphical user interface indicating a location of the 
region within the target image (Abdel col. 8 lines 45-49: The references describes displaying the results of the 
matches to a user. Since the images are displayed, and these images indicate to the user the location of the region, 
the limitations are met). 

Regarding claim 101, Abdel discloses that for at least one region of the target image that matches the color 
features of the template image, displaying information on a graphical user interface indicating a degree to which 
color information of the region matches color information of the template image (Abdel col. 8 lines 45-49: The 
reference describes listing the images in a ranked fashion). 

Regarding claim 102, Abdel discloses receiving the target image: wherein the target image is received form 
oen of the group consisting of a memory medium, a hardware device and a software application (Abdel Fig. 3). 

Regarding claim 103, Abdel discloses that either of the template image or the target image is a portion of a 
larger image (Abdel col. 1 lines 29-36: The reference describes using a single frame of a video clip (i.e. a larger 
image).) 

Regarding claim 107, Chen discloses determining a step size that is used in determining locations for the 
plurality of regions within the target image (Chen pg. 644: The reference describes determining quantization levels 
(i.e. step sizes).). 
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Regarding claim 108, Chen discloses determing sub-sampling size that is used to determine the size of the 
at least a subset of pixels examined for each of the plurality of regions of the target image (Chen pg. 644: The 
reference describes determining quantization levels that in turn determine the size of a subset of pixels examined for 
each image region. Thus, these quantization levels meet the requirements of the claim.). 

Regarding claim 1 10, Chen discloses converting either of the template image or the target image to HSI 
format (Chen pg. 644). 

Regarding claim 111, Chen discloses determing that the examined pixel can be categorized as either black, 
gray, or white based on one or more of saturation and intensity values of the respective pixel. And then assigning 
the examined pixel to a black, gray, or whtie category if the examined pixel can be categorized as black, gray, or 
white, respectively. Chen further discloses determining a color category for the examined pixel based on one or 
more of hue and saturation values of the examined pixel, if th examined pixel cannot be categorized as either black, 
gray, or white (Chen pg. 644, see heading "non-uniform quantization of HSV color space"). 

Regarding claim 1 14, Chen discloses determining one or more dominant color categories wherein the one 
or more dominant color categories are assigned a relatively larger proportion of examined pixels, with respect to 
other color categories of the color space (Chen pg. 644, see heading "non-uniform quantization of HSV color 
space"). 

7. Claims 104, 105, 106, 1 12, 1 13, 1 15 are rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Abdel and Chen as applied above, and further in view of Tao (USPN 5,799,105) 

Regarding claim 104, the combination of Abdel and Chen fail to expressly disclose that a user specifies 
search criteria. Tao, on the other hand, discloses that user input a "grading critera" (such as a hue value) to be used 
for searching and sorting the images (Tao col. 7 lines 59-63). It would have been obvious to one reasonably skilled 
in the art at the time of the invention to modify Abdel and Chen's target image locating method by having the user 
input search criteria as taught by Tao. Such a modification would have allowed for a more streamlined searching 
process that took specific user needs into consideration. 

Regarding claim 1 12, Tao discloses receving user input spectfyng a desired color sensitivity level to use in. 
locating target image regions that match the template image, wherein the user input determines a number of 
categories into which the HSI color space is divided (Tao col. 3 lines 44-58). 

Regarding claim 105, the combination of Abdel and Chen fail to expressly disclose a coarse-to-fine 
multiple pass search heuristic. Tao, on the other hand, discloses this search method (Tao col. 14: The reference 
describes running search phase I, and then running search phase II— which is a finer version of the first phase). It 
would have been obvious to one reasonably skilled in the art at the time of the invention Abdel and Chen's search 
method by incoorporating a coarse-to-fine search method as taught by Tao. Such a modification would have 
allowed for faster processing time because the most processing effort was focused at the most likely candidates. 
Such methods are commonly employed in the art. 
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Regarding claim 106, Tao discloses performing a first-pass search through the target image to find initial 
match candidate areas performing one or more subsequent search passes in which proximal regions proximal to the 
candidate areas are searched in order to find a best matching region in the proximal region (Tao col. 14). 

Regarding claim 113, Abdel and Chen fail to expressly disclose matching a template region and a target 
region, but they fail to expressly disclose comparing percentages to make that determination. Tao, on the other 
hand, discloses such a percentage comparison (Tao col. 7 line 59 - col 8 line 6). It would have been obvious to one 
reasonably skilled in the art at the time of the invention to modify Abdel and Chen by comparing percentages of 
colors as taught by Tao. Such a modification would have allowed for would have allowed an additional way of 
matching images by using colors and would have thus made for a more robust system. 

Regarding claim 115, The combination of Abdel, Chen, and Tao discloses that for each dominant color 
category, comparing the percentage of template image pixels assigned to the dominant color category to the 
percentage of target image region pixels assinged to that color category. 

8. Claims 109 and 1 16 are rejected under 35 U.S.C. 103(a) as being unpatentable over the combination of 
Abdel and Chen as applied above, and further in view of Wang et al. (Wang) (USPN 5,802,361). 

Regarding claim 109, Abdel and Chen disclose color characterization analysis of both the template and 
target images. However, Abdel and Chen fail to expressly disclose that the analysis of the target region is only with 
respect to a subset of the pixels. Wang, on the other hand, discloses analyzing a subset of the target region pixels 
(Wang col. 9, and elsewhere throughout the specification). It would have been obvious to one reasonably skilled in 
the art at the time of the invention to modify Abdel and Chen's image matching method and system by only 
searching a subset of the pixels in the target image as taught by Wang. Such a modification would have allowed for 
a more efficient system that processed less pixels while still achieving high quality results. 

Regarding claim 1 16, Abdel and Chen fail to expressly disclose the required smoothing step. Wang, on the 
other hand, does disclose this step (Wang col. 9: The reference describes a low-pass filtering operation — which is a 
smoothing operation). It would have been obvious to one reasonably skilled in the art at the time of the invention 
Abdel and Chen's image matching system and method to incorporate the smoothing effect taught by Chen. Such a 
modification would have allowed for reduced high frequency details in the image and thus allowed for the system to 
more easily detemine a match (Chen col. 9 lines 19-24). 

9. Regarding claims 1 17-151, all of the limitations of these claims have been discussed above. The above 
rejections are specificallly applied to these claims as well. The only additional limitation required by these claims is 
a processor and a memory medium. This is disclosed in the Abdel reference (see Fig. 3, generally). 
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Conclusion 



10. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. 
Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension 
of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the 
mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final 
action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, 
then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee 
pursuant to 37 CFR 1. 136(a) will be calculated from the mailing date of the advisory action. In no event, however, 
will the statutory period for reply expire later than SIX MONTHS from the date of this final action. 

1 1 . Any inquiry concerning this communication or earlier communications from the examiner should be 
directed to Patrick L Edwards whose telephone number is (571) 272-7390. The examiner can normally be reached 
on 8:30am - 5:00pm M-F. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Joe Mancuso 
can be reached on (571) 272-7695. The fax phone number for the organization where this application or proceeding 
is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent Application Information 
Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR 
or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more 
information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the 
Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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